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We define the local empirical process, based on n i.i.d. random vectors in dimension d, in 
the neighborhood of the boundary of a fixed set. Under natural conditions on the shrinking 
neighborhood, we show that, for these local empirical processes, indexed by classes of sets that 
vary with n and satisfy certain conditions, an appropriately defined uniform central limit theorem 
holds. The concept of differentiation of sets in measure is very convenient for developing the 
results. Some examples and statistical applications are also presented. 
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1. Introduction 

Let Xi,. . .,Xn be independent and identically distributed (i.i.d.) random vectors in M.'^ 
(d G N), distributed according to an absolutely continuous probability measure P. Denote 
the corresponding density by p. For a Borel measurable subset D of M'', write 

n 
i=l 

The process '^n{D),D eV {V being the class of Borel sets), is, by definition, a binomial 
process on W^; "^n/n is the empirical measure corresponding to Xi,...,X„. Clearly, 
E*„(D) = nP(Z?). 

Let Khea, convex body in R''. The set K will be fixed throughout. Denote its boundary 
by dK . It is the aim of this paper to study the behavior of ^„ in the neighborhood of 
dK . Write |jz — dK\\ = wiixedK \\z — x\\ and let 

Ve{dK)^{zeM.'^:\\z-dK\\<e}, e > 0, 
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denote this neighborhood. Set a = P{ye{dK)). For a Borel set A C V^{dK), define 

zM) = -^[^r.{A)-nP{A)]. 
^Jna 

If £ — >■ 0, then ah sets A will "shrink toward" dK. If, however, n — oo at the same 
time, the random variables Zni^A) do not have to converge to and, if ne — > oo, they 
should typically converge to Gaussian random variables. However, where would these 
Gaussian random variables "live" ? Would they form some set-parametric process? These 
are the questions we seek to address in this paper. We will do this using the concept of 
differentiability of set-valued functions, as was recently developed in [24]. 

One cannot prove a sufficiently interesting Gaussian limit theorem for z„ which is 
defined on all Borel subsets of \>^{dl<. \ Instead, one needs to consider smaller classes of 
sets. Let £ = e„ — > as n — cxj and let Ae„ be a class of measurable subsets of V^^ {dK). 
The canonical example of Ae^ is constructed as follows. Let ^ be a fixed class of Borel 
sets in R'' and define A = {K' AK : K' € /C}, where A denotes "symmetric difference". 
Now, take Ae„ = {A E A: A C V£^{dK)}. Particular cases can be found in Examples 1 
and 2 below. 

Our main result is the central limit theorem for the local empirical process near dK 
and indexed by Ag^ , 

{z„(A),Ae AJ. (1) 

Denoting the conditional probability distribution on Ve{dK) by Pe{A) — P{A)/a, we can 
also write 

z„(A) = ^[*„(A) - naP,{A)]- 



This reflects the fact that, on average, the effective sample size is equal to na, not n. We 
therefore assume, in addition to £„ — !■ 0, that 

n£„ — >■ oo as n — > oo. 

This will imply that na — > oo and ensure that the sets in Ae„ contain enough observations 
to obtain Gaussian limit behavior. 

Although very natural here, it is, in general, unusual that an empirical process is defined 
on a class of sets that depends on n. We will show that its limiting process should be 
defined on a class of subsets not of the "same" M."^, but of the cylinder dK x [—1,1]. The 
subsets in this class arc properly defined derivatives of sequences of sets, with the nth 
set an element of Ae„ ■ 

Poisson limit behavior of ^„ on V^,, (dK) has been studied in [26]. The main limit result 
there had a somewhat unusual property: it contained a functional limit theorem, but not 
a one-dimensional limit theorem for Indeed, although it showed weak convergence 
and, moreover, convergence in total variation, for the process ^E'n given on all Borel 
subsets of Ve„ (dK) , for a particular sequence of subsets , it remained unspecified 
which random variable from the limiting process the sequence *I'„(Ae^) would converge 
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to. This happened because the notion of derivative sets had not been developed at the 
time Khmaladze and Weil [26] was accepted for publication. In this paper, the situation 
is different - extracting the one-dimensional limit theorem from Theorem 1 gives the 
following statement: if the set-valued function Ag is differ entiahle in e at e ^ and 
dAg/de is its derivative (see Section 3 or [24]), then 

ZniA,J^W{dA,/de), 

where W is the set-parametric Brownian motion defined just before Lemma 1, Section 4. 

The local empirical process for one-dimensional Xi, that is, the empirical process in the 
neighborhood of a point c € M U {— oo, oo}, is a classical object in probability theory, one 
which has proven to be very valuable in statistics; see, for example, [5, 9, 10, 12, 14, 23, 29], 
the book by Csorgo and Horvath [5] and Khmaladze [23]. The one-dimensional local 
empirical process has been extended to the multivariate set-up, but, typically, only the 
neighborhood of a point c G R'' or the region outside a large sphere are considered; see, 
for example, [8, 11, 13, 30, 34]. Perhaps the closest to the present paper are [18] and [15]. 
For a local empirical process for function- valued random elements, see [16]. 

The paper is organized as follows. In the next section, we present statistical applica- 
tions. In Section 3, we introduce the necessary geometry and the appropriate concept of 
differentiation of sets. In Section 4, the main results, central limit theorems for z„, and 
some examples will be presented. Proofs are collected in Section 5. 

2. Statistical motivation 

Although the local empirical process near the boundary of a set is an interesting prob- 
abilistic object in its own right, the study of this type of process was mostly motivated 
by problems in spatial statistics. Consider a family of distributions, indexed by some 
parameter 9, and denote by Ln{9,9') the log-likelihood ratio. If the parameter were a 
vector, as in parametric problems (see, e.g., [22]), the local analysis of Ln{9,9') (or any 
other process which the inference is based upon) , in 9' from the neighborhood of the true 
value 0, is a crucial step in asymptotic statistical theory. It forms, for example, the basis 
of contiguity theory. The situation is similar when the parameter is a function (see, e.g., 
[4, 40]). However, it has thus far not been known how to carry out such a local analysis 
when the parameter is a set. 

Examples of set-parametric problems are provided by the class of spatial change point 
problems or change set problems (see, e.g., [25]). In these problems, the observation is 
usually a (marked) point process in K*^ and the model assumption is that there is a set, 
or an image, K, such that outside K, the distribution of the point process (e.g., the 
distribution of the marks) sharply changes. One can think of K as, for example, an ore 
deposit site, a pollution site or a site with different magnetic properties. The literature 
on this problem is very broad; see, for example, [6, 19, 28, 32]. 

In most of the particular formulations of the change set problem, the log-likelihood 
ratio Ln{K, K') is some form or another of the local empirical process (1), where K plays 
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the role of the true value of the change set, while the sets K' are small deviations from 
it. To be more precise, let (Xi, Fi), . . . , (X„, F„) be independent random vectors, with 
Xi being (as before) a d-dimensional location and Yi being a "mark" . not necessarily 
one-dimensional. Write K[e) instead of K' in order to explicitly express the dependence 
on e > 0. Let Pi and P2 be the distributions of Yi outside K and on K , respectively. The 
log-likelihood ratio then has the form 

n 

L^{K,K{e))^Y.^lK(e)\K{X^)-lK\K(e){Xmy^). 
1=1 

where i{Y{) = log ^(F^)- We focus on 

n n 
^'^K(e)\K{Xi) and ^lK\K(e){Xi); (2) 

2=1 1=1 

a discussion of the behavior of the ^(1^) is of secondary importance here. Let JC be 
a class of set-valued functions A'(-) all converging to the same K, that is, A'(£)A/^ 
shrinks toward the boundary dK when e — >■ 0. These sets describe the deviations from the 
hypothetical change set K . Let us consider the processes in (2) given on these deviations 
and investigate their joint limit in distribution when rt — > oo and e = £„ — > 0: the larger the 
number of observations, the smaller, or narrower, the sets we consider. In the appropriate 
formulation of "local alternatives", where not only K{e) tends to but also P2 tends to 
Pi, when n — > cxj, the convergence to a Gaussian limiting process - as shown in this paper 
- is of fundamental importance. Indeed, although there is a rich literature on statistical 
estimation of sets, we know very few results on testing hypotheses about sets and no 
results for testing against local alternatives. 

To illustrate another class of statistical problems where the parameter is a set, consider 
two prominent examples: the excess mass approach (cf. [31, 33]), and the shorth [2, 20]) 
and its generalization [17]. Let /C be a fixed subset of P, as in the canonical example. 
One could choose, for instance, K. to be the class of all ellipsoids. Define the excess mass 
set K for level A > by 

Ar = argmax{P(if') - \^Jid{K')}, 
K'eK 

where fj,d denotes d-dimensional Lebesgue measure. Similarly, the generalized shorth or 
minimum volume set K for probability a S (0, 1) is given by 

K = a.rgmm{fM{K') : P{K') > a}; 
K'eic 

see [7, 35] when K, is the class of all ellipsoids. It turns out that both of these sets K and 
their M-estimators can be analyzed somewhat similarly. Therefore, we confine ourselves 
to the excess mass set. 
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The obvious non-parametric estimator for this set is obtained by replacing P by '^n/n: 

Kr, = argmax{1'„(A")/?i - \^J^d{K')} 
K'eK 

= argmaxn2/3{^'„(A")/?^ - X^^diK') - (*„(AO/n - XfidiK))}. 

Let e„ be such that a — n^^^"^ (cf. the "cube root asymptotics" of [27]). Under certain 
conditions, it can be shown that for large T > 0, with high probability, Kn^K G ^Te„ ■ 
Observe that in that case, 

Kn = argmax {^„(X') - z^iK) + n''/''[P{K') - P{K) - \{^Xd{K') - ^ld{K))]}. 

Now, a central limit theorem for z„ makes it possible to show that, asymptotically, 
Kn^K can be described as a deterministic function depending on n (actually, on t~^; 
see Section 4) evaluated at a random variable that does not depend on n. This random 
variable is the argmax of some Brownian motion with drift. Such a result is very useful 
for a refined analysis of Kn ■ See [3] for a study of the behavior of such A'„ 's along these 
lines. 

3. Some geometry and differentiability of sets 

In this section, we first briefly review some relevant notation and facts from geometry. 
We then recall the concept of "differentiation of sets in measure" , as given in [24] . In 
that paper and the references therein (in particular [36]), more details about the required 
geometry can be found. We also refer to the recent monograph [37]. 

Let K G 2? be our convex body, that is, a closed, bounded convex set that has interior 
points. Denote by n(z) the metric projection of z G R'' on dK, that is, Il{z) is a nearest 
point to z on dK . The set of z-valucs for which such a nearest point is not unique is a 
subset Sk of K called the skeleton of K . Let fi^ denote c?-dimensional Lebesgue measure. 
It is then known that fJ-diSx) = 0. A unit vector u is called an outer normal of K at 
X G dK if there is some z G R'^\A' such that x = n(z) and u = {z — n(z))/||z — n(z)||. 
Let Br{z) denote the closed ball with center z and radius r. For x G dK, wc define the 
local interior reach 

r{x) = max{r : x G Br{z) C AT}. 

If r{x) > 0, then the outer normal u at x G dK is unique. In this case, the unit vector 
—u is the unique inner normal. In general, at each x G dK, we denote the set of outer 
normals by N{x) and the normal bundle of K is defined as 

Nor{K) = {{x,u):xe dK, u G N{x)}. 

The cylinder S = Nor{K) x [—1,1] will be important for describing our limiting processes. 
Note, however, that it will eventually be possible to work with the cylinder dK x [—1, 1], 
which is much easier to visualize. 
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We also need the so-called local magnification map r^; see [24]. Any point z e W^\Sk 
can be written as z = n(z) + ds{z)u, where ds{z) is the signed ("+" outside) distance 
between z and n(z) and u an outer normal at n(^) that satisfies the equality. Now, 
define 

re(z) = (^n(z),u,^^^, zeM.\SK,e>Q. 

Observe that maps V^{dK)\SK into E. 

We are now prepared to introduce the aforementioned differentiation of sets. Consider 
the first support measure dd-i on Nor{K)\ see [36]. It attributes measure to the set of 
all points where, at a;, there is more than one outer normal u. Hence, we can map 

it to dK in a one-to-one way. On dK ^ this map coincides with Hausdorff measure v and 
if, for a Borel set H C Nor{K), we write 

Ho^{xedK:{x,u)eH}, 

then 

dd-i{H)^u{Ho). (3) 

On E, define the measure M = ^d-i x (/i being onc-dimcnsional Lcbcsgue measure). 
Consider a (Borel) set- valued function K{e), e € [0, 1], such that K{0) = K, with K as 
before; write A{e) = K{e)AK and assume that A{e) C Ve{dK). The set-valued function 
A{e), e G [0, 1], is called differ entiahle at dK and £ = if there exists a Borel set B gT, 
such that M(TeA(e)AB) -> as e -> (where t^A = {ts{z) : z e A}). The set B is called 
the derivative of A{£) at dK . In this case, we also say that i^(e),£ € [0, 1], is differentiable 
with the same derivative and write 



=0 de 



= B. 

e=0 



Note that B is not unique, but can be changed on a set of M-measure 0. 

Let P now be as in Section 1. We require that the density p can be approximated in 
the neighborhood of dK by a function depending only on n(z) and on whether or not 
z G K . This latter possibility is easy to imagine in the change set problems: the limit of 
p{z) from inside K can indeed be different from that from outside if K^ is the change set. 
More formally, we require the existence of two functions, p^ and on dK such that, 
as e — )■ 0, 



|p(z)-p+(n(z))|dMd(z)^0, (4) 

V,idK)\K 

\p(z)-p^{U{z))\d^ld{z)^0■ (5) 

V^_{dK)nK 



Now, define a measure Mp on E as follows: 

dMp(x, u, s) — p+{x) d-dd-iix, u) x ds for s > 0, 
dMp(x, u, s) = P-{x) ddd-i{x, u) x ds for s < 0. 
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For convenience, assume that p+ and p- are bounded (although a weaker, integrabil- 
ity, condition would suffice). An easy, but practically interesting, situation occurs when 
p+{x) ~ c+ and P-{x) = c_ for all x G dK, where c+,c_ > are two constants. 

The following key result from [24] shows the "differentiability of sets in measure": if 
A{e) is differentiable at dK, then 

)■ (6) 

£ = 0/ 



de 



P{A{e)) 



F = 



4. Main results 



Let Ae„ be as in Section 1 and assume Mp(E) > 0. Writing a„ = P(Ve,, (i9X)), it easily 
follows, using (6), that a„/e„ — >■ Mp(S). Hence, we have, just as for e„, 

a„ — > and na„ — >■ oo. 

Denote by B the class of all possible derivatives at e = corresponding to Ae^ , which, 
by definition, means that B £ B if and only if there exists a sequence of sets (^n)^i 
with An e Ae„ and M{T^^An^B) 0. (Observe that for a thus converging sequence 
of Borcl subsets of E, the limit set is not well defined. This limit "set" is actually an 
equivalence class of sets, defined by the property that for any two sets Bi , B2 in the class, 
M{BiAB2) = 0. Out of every such an equivalence class, we choose one limit (Borel) set 
B, say. Whether or not the conditions of our results are satisfied will depend on the 
choices of these i?'s. In applications/examples, we should choose natural or appropriate 
B's to make the theorems work.) 

Consider the local empirical process (1) from Section 1. To establish its limit in distri- 
bution, we need the following steps. Write t~^C = {z G Ve{dK) :Tf{z) € C} for a Borel 
set C C S. First, using the local magnification map, induce the point process and the 
distribution Q„ on E : $„(C) = ^n{T~J^C) and Q„(C) = Pe„ {t~J-C). Thus, for any Borel 
set C C E, we can define 



Vn{C) := -^[$„(C) - naMC)] 




(7) 



[*n(r. ^C) - na„P,„ (r.^C)] ^ z,,{t-^C). 



Hence, we can define the local empirical processes on two classes of sets: on S„ := 
\t^^A:A £ vAe^}, which changes with n, and on B, the class of its limits, or derivative 
sets, which is fixed. We denote these processes by 

w„3„ :={w„(S):Sei3„} and w„,b := {w„(S) : B e 6}. 

(Local empirical processes of the latter type - i.e. for a fixed B - have been studied in, e.g., 
[15, 18]. Here, however, our main object is Vn,B„', Vn,B is an auxiliary process, a bridge 
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between Vn,Bn ^^'^ its limiting process.) Second, we show that the distribution (5„, which 
"governs" these processes, converges to the distribution Q{C) = Mp{C) / Mp{Y) and the 
processes VnS„ can be approximated by the processes Vn,B- Next, we verify that Vn,B 
converges in distribution to a set-parametric Brownian motion Wq and, finally, we note 
that one can switch from VFg, given on subsets of S, to its isometric image, given on the 
"easier" cylinder dK x [—1, 1]. 

Below, we write C_ = {(a;,u,s) € C:{x,u) € Nor{K),s< 0} and C+ = C\C_. Note 
that in the case p+{x) = c+ and P-{x) = c_ for all x G dK , we have 

_ c+A/(C+)+c_M(C_) 
' {c++c^)y{dK) ■ 

When, for example, = c_ < c+, we obtain Q(C) = Q{C+) = M{C+)/v{dK). 

For Borel sets C, C" C S, define d{C, C) = (g(CAC"))^/^ Throughout, we wiU assume 
that (B, d) is totally bounded and that 

sup inf d{T, A,B)^0. (8) 

In particular, every sequence (A„)5^j with An € Ae^ has a subsequence (^nfc)fcLi such 
that for some B gB, d{T^^^An^,B) — > 0. Assumption (8) can be written as 

sup inf d(Bn,B) 0. 

B„eB„BGB 

From the definition of B and the assumption that (B, d) is totally bounded, it follows 
that 

sup inf d(Bn,B) ^Q. 

BeBBr^&B,, 

Thus, the Hausdorff distance between the classes Bn and B tends to 0: 

7„:=max( sup inf d(i3„, S), sup inf d{Bn,B)]^0. (9) 

^B„eB„BeB BeBB^eB„ J 

Recall that it is the aim of this paper to present a central limit theorem for Zn^Ann ' 
cquivalcntly, Wn,B„- By "central limit theorem for Zn^As,^" mean: 

(a) sup \Vn{Bn) - VniB)\ ~> 0; 

B„<£Bn,BeB;d{B„,B)<'Y„ 



and 



(b) Vn.,B^WB:^{W{B),BeB}. 



Here, Wg is set-parametric Brownian motion: a bounded, uniformly c?-continuous Gaus- 
sian process with mean and covariance structure E,W{B)W{B') = Q{B D B'). We view 
Vn and W as processes taking values in £°°{B) endowed with the uniform distance and 
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understand weak convergence in the sense of van der Vaart and Wellner [39]. (We as- 
sume, for convenience, that our classes of sets are such that the various "suprema" are 
measurable, i.e., that they are random variables.) The following fact is very useful for 
proving this central limit theorem. 

Lemma 1. From (4) and (5), it follows that Qn converges to Q in total variation: 



with the sup taken over all Borel sets C C S. 

Define dr^{A,A') (Pe„ (AAyl'))^/^ (P(AAA')/a„)i/2; observe that dn{A,A') = 
iQn{Te„AATe„A')y^'^. Assume, for any S > 0, that there exists a finite collection of pairs 
(brackete) [^(J), A((5)] of Borel sets in Ve„{dK) with dniA{S) ,A{S)) < (5,juch that any 
set A G Ae„ can be placed in a bracket from this collection: A{S) C Ac A{S). Consider 
such a class of brackets with minimal cardinality; denote this cardinality (the bracketing 
number) by 7V[].„(5) and let A/j]_„((5) be the set of A{S)'s in this class. We assume the 
same for t~^B := {t~^B:B £ B} and use the notation TV]] ^((5) and A/'[]_„((5). We will 



Theorem 1. Under the aforementioned assumptions, in particular, the growth condi- 
tions on En, the approximation of p by p^ or p^ in (4) and (5), the relation between Ag^ 
and B specified in (8) and the entropy conditions (10) and (11), the central limit theorem 
for Zn,Ae^ holds, that is, statements (a) and (b) hold true. 

We also present a version of Theorem 1 without assuming bracketing conditions. To be 
more precise, we will assume that our classes of sets near dK are Vapnik-Chervonenkis 
(VC) classes (see, e.g., [39], Section 2.6, for definition and properties). 



Theorem 2. Let Ae^ be a VC class with index tn < t for some t G N; also, assume that 
B is a VC class. If we assume that £„ — > 0, ne„ — > oo and (4), (5) and (8), then the 
central limit theorem for z„^^^^ holds, that is, statements (a) and (b) hold true. 

Remark 1. Consider the canonical example of Section 1 and let /C be a VC class. Then 
A is also a VC class, with index t, say. Since A^,-, C A, the index i„ of Ae„ indeed satisfies 



sup|g„(C)-Q(C)|-^0, 



require 




(10) 




(11) 



tn<t. 
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Remark 2. Similar to the discussions in [24, 26], we note that Theorems 1 and 2, as 
well as the whole construction, can be carried over to the case where is a finite union 
of convex bodies and, even more easily, to the case where K is closed and bounded and 
has a boundary of positive reach (intuitively, K has a "smooth" boundary). Indeed, the 
key objects, such as the local magnification map (uniquely defined almost everywhere 
on W^), the local Steiner formula, the notion of derivative sets and Lemma 1, are all valid 
for such a K . Moreover, the existence of the local Steiner formula for a very general K 
has been demonstrated in [21]. This offers perspectives for considering such a general K 
in the statements of our results. 



The limiting process Wg is defined on subsets of the cylinder E = Nor{K) x [—1,1]. 
This cylinder is not easy to visualize. However, since the support measure depends 
on H only through iJp (cf. (3)), we have a similar result for the measure Q. That is, if 
we write, for a Borcl set C C S, 

Co = {(x,s)Gaif X [-l,l]:(a;,w,s)eC} 

and we use the same letter Q for the measure 

dQ{x s) = P±(a:)d:^(x) x ds for s ^ 

which lives on dK x [—1,1], then 

g(Co) = g(c). 

Therefore, if convenient, we will replace E by F = dK x [—1, 1] and replace We with the 
process Wbo defined on So = {i?o '■ B E B}, a class of subsets of F. However, we could not 
do this with . 

Weak convergence in function spaces is important because of its statistical application, 
the continuous mapping theorem, which states that continuous functionals, or statistics, 
of the random elements involved inherit the weak convergence. We now formulate a 
continuous mapping theorem in our unusual setting, where the domain of the functions 
depends on ??. Let £°°{Bn) and £°°{B) be the spaces of bounded functions on Bn and 
B, respectively; let x„ £ £°°{Bn), x G £^{B) and assume that x is d-continuous. Also, 
assume the functionals ipn-^°"{Bn) — >■ ^ and ip:£°°{B) — )- K are such that (with 7„ as 
in (9)) 

sup \xr,{Bn)-x{B)\^Q (12) 

B„ee„,Se6;rf(B„,B)<7„ 

implies 

We then have 

fn{Vn,B^)^ V{Wb)- (13) 
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sup \vniB^)\ A snp \WiB)\. 

B„eB„ BeB 

For the proof of (13), we only mention that a Skorokhod almost sure representation 
theorem yields the existence of Vn,B = '^n.B a-nd Wb Wb such that 

sup |u„(B)- 1^(5)1^0 a.s. 

If we extend Vn,B to Bm we obtain, from (a), 

sup \v,,{B^)~W{B)\ 

B„eB„,SeB;d(B„,B)<7„ 

< sup \Vn{Bn) - Vn{B)\ 

B„GB„,BGB;d(_B„,B)<7„ 

+ sup|5„(B)-T^(S)|4o. 

B£B 

Now, compare this with (12). The rest of the proof is elementary. 

Example 1. Let K = {(x,y) S : + < 1} be the unit disc, so dK = {(a;,?/) G 
: + _ jg ^Yie unit circle. We have Sk = {(0, 0)} and r{x) = 1 for all x € dK. 
Also, Ve(aA') = {(z, y)eM.^:{l- e)^ < + < (1 + e)^}. 

(a) Let £ be the VC class of all closed ellipses (with interior) in M^. This £ is an example 
of the general JC in the canonical example in Section 1. Thus, A= {EAK:Eg £} and 
Ae„ ={AeA:AcVe„ {dK)}. By Remark 2, Ae„ is a VC class with uniformly bounded 
index. 

We parametrize dK with the angle 9 E [0,27t) and re-express the cylinder F = dK x 
[—1,1] as [0,27t) X [—1,1]. Consider the functions fa,a,b,cM' [0,27t)^ [^1:1]i defined by 

fa,a,b,cA^) = -fW = a + fesin^(6' - a) + csin(6' - a) + dcos{6 - a), 

with a G [0, 7t/2) and a, 6, c, d G M such that supo<9<27t \ fo,a,b,c,d{0)\ < 1. Denote the class 
of all such functions hy J-£. A tedious calculation shows that 

Bo = {{(0, y) G [0, 27T) X [-1, l]:0<y< .f{e) or f{e) < y < 0} : / G J-^}- 

Since Bq is a limit class, it can be shown, directly using the definition of a VC class, that 
So is also a VC class. For B G Bq, note that for every 9 G [0,27t), the intersection of B 
with {{9,y):y E [—1,1]} is convex (an interval). Part (b) shows that this need not be the 
case in general. 

(b) Consider, for the same K, the very simple class 

^{{zeM.^:\\z-dK\\/ene [a,b]U [c,d]} : -I < a < b < c < d < 1}. 
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Now, 

■So = {{(6', 2/) e [0,27t) X [a, 6] U [c, d]} : -1 < a < 6 < c < d < 1}. 

Here, i5„ ~ B. 

Example 2. Let K = {(.t, y) £ : < x, y < 1} be the unit square with boundary 
dK. We obtain Sk = {(x, x) : < cc < 1} U {(x, 1 — x) : < x < 1} and for, for example, 
{(a;,0) :0 < x < 1} C dK , we see that r((x,0)) = min(x, 1 — x). It is notationaUy some- 
what cumbersome to describe Ve{dK) exphcitly, but it is trivial to see that it is the 
difference of a set which is a "square with circular corners" and a smaller square. 

(a) Let Q be the VC class of all closed quadrangles in . Set A = {Q/S.K :Q & Q] and 
Ae„ = {A E A:A C Ve„{dK)}. Again by Remark 2, Ae„ is a VC class with uniformly 
bounded index. The present example is somewhat similar to Example 1, but there is a 
substantial difference since a square is less smooth than a disc. 

We parametrize dK with 6* G [0,4), the counterclockwise "distance" from the origin, 
and re-express the cylinder F as [0,4) x [—1,1]. Consider the functions /Q^f,:[0,4) — )■ 
[—1, 1], with a = (ao, oi, 02, 03) and b = (&o, ^3), defined by 

fa,b{^) = fW = arn{e - m) + ioY m < 9 < m + 1 , m = 0, 1, 2, 3, 

with a,b such that a™ E [~2,2] and supo<5)<4 |/a,6(6')| < 1. Denote the class of all such 
functions by Note that / £ J^q is typically discontinuous, in contrast to an / G J-'g 
of Example 1. It can be shown that 

Bo = {{{0, zj) e [0, 4) X [-1, 1] : < 2/ < f{9) or f{0) < y < 0} : f E Tq}. 

It readily follows that Bq is a VC class. 

(b) Consider (for the same K) a larger class than Q, namely C, the class of all convex 
bodies in K^. For convenience, let P be the uniform distribution on [—1,2]^. The class C 
is again an example of the general /C in the canonical example in Section 1, but it is not 
a VC class. We have A = {CAK -.C E C} and Ae„ ^ {Ae A: A c V,^ (dK)}. 

Consider the functions /: [0,4) [—1, 1] defined by 

f{0) = fm{0 -m) for m<6l<m + l,m = 0,l,2,3, 

with /„! : [0, 1) [—1, 1] a concave function. Denote the class of all such functions by Fe- 
lt can be shown that 

6o = {{{0, y) e [0, 4) X [-1, 1] : < y < f{e) or /(0) <y <Q} : f E Fc}. 

The conditions of Theorem 1 are satisfied. In particular, using [39], Corollary 2.7.9, it 
can be deduced that (10) and (11) hold true. 
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Proof of Lemma 1. Based on the local Steiner formula, in the proof of Theo- 
rem 2 of [26], it is shown that the measure P(r~^^-)/e„ converges in total varia- 
tion to the measure Alp. This implies that P{Ve„{dK)) /sn — > A/p(I]) and hence that 
Qn = P(T-i-))/P(Ve„ (dK)) converges in total variation to Q = Mp/Mp{Y). □ 

Proof of Theorem 1. First, we prove statement (a): 

sup \Vn{B (i?)|4o. 

B„eB„,BeB;d(B„.B)<7„ 

From relation (7), Lemma 1 and the Markov inequality, it follows that it is sufficient to 
show that 

limlimsupE sup \zn{A) ~ Zn{A)\^Q. (14) 

d„(A,A)<(5 

We use [38], Lemma 19.34, page 286, for the proof of (14); in that lemma, we choose 
the indexing functions to be 1^ — 1^- We then obtain, taking the 5 there to be equal to 
5yfa^, that for some constant c, 

E sup \Zn{A) - Zn{A)\ 

d^{A,A)<5 



1 



+ ^ f 1 . AP\ 

Using na„ — >■ oo and (10), (11), we see that the second term on the right is equal to for 
small S and large n. The first term is easily seen to be bounded by 



^/^ogl\Jx) dx + c / y^logiV[]_„(a;) dx. 



Hence, (14) follows using (10) and (11). 

For a proof of statement (b) , we need weak convergence of the finite-dimensional dis- 
tributions and tightness of Vn.B- The weak convergence of the finite-dimensional distribu- 
tions follows easily from Lemma 1 and an appropriate version of the multivariate central 
limit theorem. 

To prove tightness, we use [39], Theorem 2.11.9, a general bracketing central limit 
theorem. We will choose d for the semimetric p on B which is required in that theorem. 
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For tightness, three conditions have to be fulfihed. The first one holds trivially since 
is a sum of indicators. The third one follows readily since it is essentially our condition 
(11). It remains to show the second condition: 

" / 1 1 \^ 

sup ^E^^=l^-^i5(X,)-^=l,-jB,(X,) ^0 for every <5„ i 0. 



d{B,B')<S„ * 

However, 

1 " 

no-n d{B,B')<5„ -^^ 

= — sup P{t^^\BAB'))= sup Qn{BAB'). 

O'fi d{B,B')<S„ " Q{BAB')<51 

Lemma 1 now immediately yields s„ — > 0. □ 

Proof of Theorem 2. Again, we first prove statement (a) and note that it suffices to 
show, for any jy > 0, that for i5 > small enough and then for large n, 

P( sup \zn{A) - z^{A)\ > 2t]) <2r]. (15) 

d„(A,A)<VS 

We have, for n large enough, 

P( sup |z„(A)-z„(i)| >27;) 
^AeA^„,AeT-^B ' 

d^{A,A)<s/T> 



= P( sup |z„(A)-z„(i)| >27/) 

A(^A^^,A(^r^^B ' 
P(AAA)<(5a„ 

= P( sup W{A\A) - z,,{A\A)\ > 2?7 

A&Ae,^,AeT^^B 

P{AAA)<Sa„ 

<P( sup |z„(A\i)| >ry) 

AeA,„,A<ETr,!B 
P{AAA)<Sa„ 

+ P( sup |z„(i\^)| >77) 

^AeA^^,AeT-^^B ' 

P{AAA)<8a^ 

<2P( sup |z„(C)|>77), 



(16) 
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where C„ = {A\A : A e Ae„ , A g t'J^B} U {A\A : A G Ae„ , A e t;;^^}. It can be shown 
(see, e.g., [39], page 147), using Ai\A2 = Ai n Aj, that d is a VC class. Also, the index 
Wn of this VC class is bounded: max„gN w„ < oo. 

We have, writing N = 5'„(Ve„ (SX)) and k = ria„, that 



sup |z„(C)|>7; 

^CeC„,P(C)<(5a„ 



> P( sup \Zn = TO 



sup 



2=0 \CGC„,P(C)<(5a, 



Vk 



['fniC)-nP{C)] 



sup 

C<£Cn,P{C)<5ar. 



+ ¥{\N^k\>C„Vk), 



Vk 



where Crj is chosen such that the latter probability concerning the binoniial(n, fc/n) 
random variable N is bounded by rj/2 for large n. Hence, for large n, 



i sup \ZniC)\>7j) 

^CeC„,P(C)<Sa„ ^ 

m— [k+c^Vk] / 



^ E : 

m— lk—C\i V^] 

+ E : 

m=rfc-C„\/I] 



sup 

, cec„,Pe„(C)<(S 



sup 

cec„,Pe„(c)<<5 



> 



P(Af = TO) (17) 



l=\m-k\P,^{C)>^]FiN^m) + :j^, 



where the Yj are i.i.d. random vectors on V^^ (dK) distributed according to P^^ . Note 
that in the first probability of the second sum, no randomness is involved and that this 
sum is equal to for S small enough. For the first sum, we need a good bound for 
exceedance probabilities for the suprcmum of the empirical process on a VC class. We 
will use [1], Corollary 2.9. Using max„gN w„ < oo, this leads to the following upper bound 
for the left-hand side of (17): 



m— lk+C\Vk\ 



16exp(-77V(36(5))P(A^ = m) + ^< 16cxp(-7;V(36(5)) + ^<V 



for small enough 5. So, because of (16), we have proven (15) and hence (a). 



560 



J.H.J. Einmahl and E.V. Khmaladze 



For a proof of (b), we only need to show tightness of Vn,B since the weak convergence 
of the finite-dimensional distributions follows as in the proof of Theorem 1. 
For proving tightness, we need that, for any 77 > 0, 

limlimsupPf sup |w„(-B) - Vn{B')\ > 77 ) = 

<5iO n-s-oo ^ B,B'<£B ' 
d{B,B')<S 

(see, e.g., [39], Theorem 1.5.7). Again, from (7) and Lemma 1, it suffices to show that 
limlimsupPf sup |z„(A) - z„(A')| > ?;) = 0. (18) 

(i„(A, A')<2(5 

The proof of (18) can be given along the same lines as the proof of (15). □ 
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